Papers with spatial understanding
EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models (2024.acl-short)
Copied to clipboard
| Challenge: | Recent studies have revealed significant deficiencies of LVLMs in understanding visual contents, leaving the gap between current embodied intelligence and large vision-language models (LVLM) . |
| Approach: | They propose to use a benchmark to evaluate LVLMs' spatial understanding of embodied environments to evaluate their ability to understand visual contents. |
| Outcome: | The proposed benchmark is derived from embodied scenes and covers 6 spatial relationships from an egocentric perspective. |
Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers (2024.naacl-short)
Copied to clipboard
| Challenge: | Recent approaches for developing vision and language models leverage existing vision and a language expert and try to learn a mapping between them. |
| Approach: | They propose to use a resampler module to create a ‘visual prompt’ which is then fed to the large language models (LLM) using a textual prompt. |
| Outcome: | The proposed method has been shown to be effective across coarse-grained tasks like image captioning and visual question answering, but more fine-grounded tasks that require spatial understanding have not been thoroughly examined. |
Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning (2025.findings-emnlp)
Copied to clipboard
Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, Jinhua Zhao
| Challenge: | Currently, vision-language models excel in many downstream tasks but struggle with spatial reasoning, which is crucial for navigation and interaction with physical environments. |
| Approach: | They propose a framework that generates synthetic data to provide targeted supervision for VLMs across these basic spatial capabilities. |
| Outcome: | The proposed framework disentangles 2D spatial reasoning into three core components: direction comprehension, distance estimation, and localization. |
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the spatial reasoning capabilities of modern language models (LMs) however, there has been limited research into the spatial thinking capabilities of LMs. |
| Approach: | They propose a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work. |
| Outcome: | The proposed method significantly improves LMs' ability on spatial understanding, which in turn helps solve two external datasets, bAbI, and boolQ. |
GER-LLM: Efficient and Effective Geospatial Entity Resolution with Large Language Model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for integrating spatial data from diverse sources are limited by their reliance on large amounts of training data and their inability to incorporate commonsense knowledge. |
| Approach: | They propose a framework that integrates large language models into the GER pipeline. |
| Outcome: | The proposed framework improves on real-world geospatial datasets and shows that it is more efficient than state-of-the-art methods. |
Response Wide Shut? Surprising Observations in Basic Vision Language Model Capabilities (2025.acl-long)
Copied to clipboard
| Challenge: | Vision-language Models have been shown to be highly capable but lacking basic visual understanding skills. |
| Approach: | They propose to examine the limitations of vision-language models on visual tasks by constructing a series of tests that probe which components of design may be lacking. |
| Outcome: | The proposed tests compare VLMs to other models on visual encoders, intermediate vision-language projection and LLM-decoder outputs. |
VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions (2025.emnlp-main)
Copied to clipboard
| Challenge: | VEHME is a vision language model for assessing handwritten math answers . traditional methods of assessing student work are limited by time constraints, class sizes and cognitive load . |
| Approach: | They propose a Vision-Language Model for Evaluating Handwritten Mathematics Expressions to assess handwritten math responses with high accuracy and interpretable reasoning traces. |
| Outcome: | VEHME achieves state-of-the-art performance among open-source models and approaches accuracy of proprietary systems. |
Where the Cat Sat: A Multilingual Framework for Spatial Language Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work exhibits biases toward English and prepositional marking . Existing models are limited in understanding spatial relations across typologically diverse languages . |
| Approach: | They propose a multilingual framework and benchmark for spatial language understanding . they decompose spatial relations into surface elements and semantic components . their results suggest surface parsing does not entail spatial understanding - they argue . |
| Outcome: | The proposed framework and benchmark decomposes spatial relations into surface elements and semantic components. |
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks (2026.acl-long)
Copied to clipboard
Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, Yueting Zhuang
| Challenge: | Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored. |
| Approach: | They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes. |
| Outcome: | The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks. |